Speech/gesture interface to a visual computing environment for molecular biologists
نویسندگان
چکیده
Recent progress in 3-D, immersive display and virtual reality (VR) technologies has made possible many exciting applications, for example, interactive visualization of complex scientiic data. To fully exploit this potential there is a need for \natural" interfaces that allow the manipulation of such displays without cumbersome attachments. In this paper we describe the use of visual hand gesture analysis and speech recognition for developing a speech/gesture interface for controlling a 3-D display. The interface enhances an existing application, VMD, which is a VR visual computing environment for molecular biologists. The free hand gestures are used for manipulating the 3-D graphical display together with a set of speech commands. We describe the visual gesture analysis and the speech analysis techniques used in developing this interface. The dual modality of speech/gesture is found to greatly aid the interaction capability .
منابع مشابه
Invited Speech: "Gestural Interface to a visual computing Environment for Molecular biologists"
In recent years there has been tremendous progress in 3D, immersive display and virtual reality (VR) technologies. Scientific visualization of data is one of many applications that has benefited from this progress. To fully exploit the potential of these applications in the new environment there is a need for “natural” interfaces that allow the manipulation of such displays without burdensome a...
متن کاملSpeech / Gesture Interface t o a Visual Computing Environment for
Recent progress in 3-0. immers i ve display and virtual reality (VR) technologies has m a d e possible m a n y exci t ing applications, f o r example? interactive visualization of complex scient i f ic data. To fu l l y exploit this potent ial there i s a need for ‘hatural” interfaces tha t allow the man ipu la t ion of such displays wi thout cumbersome at tachments . In this paper we describe ...
متن کاملSpeech/Gesture Interface to a Visual-Computing Environment
Although recent years have witnessed tremendous progress in 3D immersive display and virtual reality, the corresponding humancomputer interaction (HCI) technologies have lagged behind. Current interfaces involve heavy headsets, datagloves, tethers, and other virtual reality devices, which may deter or distract the user. To fully exploit VR’s potential for visualizing and interacting with comple...
متن کاملUsing Handwriting and Gesture Recognition to Correct Speech Recognition Errors
To provide a natural interface to the computer, we have developed an integrated speech, gesture, and handwriting recognition system. By integrating the technologies, we can easily accomplish, most, if not all, of the tasks that are needed to use computers and their applications, including many tasks which neither technology can adequately perform by itself. This integrated environment not only ...
متن کاملDecision Theoretic Resource Management for Intelligent En- vironments
Motivation: Natural human-computer interaction in an intelligent environment requires a rich multi-modal interface. Ideally, the environment should allow user interaction via speech, gesture, movement, and other means. Similarly, the environment should speak and give visual responses, at the very least. Achieving such interaction abilities is complicated, involving many distinct hardware and so...
متن کامل